test: remove remaining SENTRY_TEST_FATAL macro usages #5624
Conversation
armcknight
requested review from
philipphofmann,
philprime,
noahsmartin and
itaybre
as code owners
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5624 +/- ##
=============================================
+ Coverage 86.125% 86.455% +0.329%
=============================================
Files 419 419
Lines 35606 35637 +31
Branches 15092 15222 +130
=============================================
+ Hits 30666 30810 +144
+ Misses 4899 4785 -114
- Partials 41 42 +1
... and 22 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
|
Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| fd5961e | 1210.59 ms | 1235.57 ms | 24.98 ms |
| 035974f | 1225.89 ms | 1251.23 ms | 25.34 ms |
| 2b02431 | 1229.63 ms | 1248.98 ms | 19.35 ms |
| e0424b9 | 1204.23 ms | 1241.08 ms | 36.85 ms |
| 2481950 | 1221.04 ms | 1248.98 ms | 27.94 ms |
| d05d866 | 1211.78 ms | 1230.96 ms | 19.18 ms |
| e64d3d4 | 1241.90 ms | 1260.10 ms | 18.20 ms |
| acac774 | 1217.76 ms | 1253.29 ms | 35.52 ms |
| 4d264fa | 1223.48 ms | 1246.91 ms | 23.44 ms |
| 884b224 | 1221.11 ms | 1255.88 ms | 34.77 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| fd5961e | 23.74 KiB | 874.07 KiB | 850.32 KiB |
| 035974f | 23.74 KiB | 874.07 KiB | 850.33 KiB |
| 2b02431 | 23.75 KiB | 850.73 KiB | 826.98 KiB |
| e0424b9 | 23.74 KiB | 874.07 KiB | 850.33 KiB |
| 2481950 | 23.74 KiB | 872.74 KiB | 849.00 KiB |
| d05d866 | 23.75 KiB | 878.60 KiB | 854.85 KiB |
| e64d3d4 | 23.75 KiB | 855.37 KiB | 831.62 KiB |
| acac774 | 23.75 KiB | 866.51 KiB | 842.76 KiB |
| 4d264fa | 23.74 KiB | 874.07 KiB | 850.33 KiB |
| 884b224 | 23.75 KiB | 879.55 KiB | 855.80 KiB |
|
@itaybre that is true, but again see my point about taking down the test suite with a crash due to abort. the test should still fail if we hit these situations, but we should be able to run the whole suite and see all test failures, instead of having to rerun every time we fix a crasher. that is much more time (and morale) consuming. especially in CI, if for some reason something happens there but not locally. |
There was a problem hiding this comment.
Thanks for following up and removing these @armcknight 💯
To be fair, I am not sure if this is something I would like to remove.
The real reason behind why these tests fail is because we can't setup the context properly.
Yes, exactly as @armcknight pointed out. We should write tests that fail when we encounter these edge cases.
It was brought up that these are disruptive to the test processes, and I actually agree with that as of previously tried to remove as many crashes as possible from the tests. Remove these and replace them with error logs, as was done in #5363
#skip-changelog